A Comparative Study on Confidence Measure in Mandarin Command Word Recognition

Two categories of Confidence Measure (CM) approaches for Mandarin command word recognition, i.e., Likelihood Ratio Testing (LRT) based CM and Word Posterior Probability (WPP) based CM, are investigated in this paper. Both Equal Error Rate (EER) and Confidence Error Rate (CER) performances of these approaches are evaluated on two databases: A Mandarin telephone command word database for which a matched model can be trained, and a PDA command word database for which the model and testing environment are mismatched. Experimental results show that for the matched case, the WPP based CM outperforms LRT based CM, and it is also insensitive to the decision threshold. However, for the mismatched case, the performance of the WPP based method decreases dramatically because of the inaccurate estimate of posterior probabilities on the basis of the ill-formed word graph. The characteristics and robustness problems of these CM approaches are analyzed, and their performances are given when deployed to realistic tasks.